Can distributed delays perfectly stabilize dynamical networks? 
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Signal transmission delays tend to destabilize dynamical networks leading to oscillation, but their 
dispersion contributes oppositely toward stabilization. We analyze an integro-difTerential equation 
that describes the collective dynamics of a neural network with distributed signal delays. With the 
gamma distributed delays less dispersed than exponential distribution, the system exhibits reentrant 
phenomena, in which the stability is once lost but then recovered as the mean delay is increased. 
With delays dispersed more highly than exponential, the system never destabilizes. 

PACS numbers: 05.45.Xt, 02.30.Ks, 87.18.Sii 
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Oscillation is not omnipresent in the brain, but is ob- 
served only in limited areas such as the cat visual cortex 
and the olfactory bulb Extensive synchronization 
occurs in rather pathological conditions such as parkin- 
sonian tremor or epilepsy ^] . How is the collective oscil- 
lation avoided in the brain composed of neurons that are 
prone to synchronize? 

Recently it has been revealed that distributed delays 
in the signal transmission contribute toward stabiliza- 
tion, in neural networks [1, 01 , ecological systems [1, @ , 
control engineering 0], biology [1] or coupled dynamical 
systems [9|. In the light of this knowledge, wide distri- 
butions of neuronal signal transmission delays reported 
recently (10.] bear apparent consistency with the absence 
of extensive oscillation in the brain. 

Conversely, the signal transmission delay itself is 
known to be a destabilizing factor We therefore 

wish to comprehend how these factors are competing in 
real dynamical systems; in particular, how the macro- 
scopic stability of dynamical systems is controlled by the 
dispersion of delays and their average. 

For this purpose, we examine the stability of an 
integro-differential equation [l^ derived from micro- 
scopic dynamics of a neural network whose signal trans- 
mission delays are distributed in time. It is revealed from 
the analysis that the network with gamma distributed de- 
lays less dispersed than exponential distribution exhibits 
reentrant stability; the system once destabilizes but then 
recovers the stability as the average delay is increased. 
With delays dispersed more highly than exponential dis- 
tribution, the system attains a perfect macroscopic sta- 
tionarity. 

We consider a network of model neurons that obey the 
evolution equation, 



_ dxi{t) 
dt 



-Xi{t) + sgn '^WijXj{t - dij) + 



(1) 



where sgn(w) is the sign function that takes values -f 1 and 
— 1 respectively for v > and v < 0. r is the "membrane 
time constant" of an individual neuron. The "synap- 



tic weight" 



and the signal transmission delay dij 



ith neuron (Fig[T|). Si is the "external stimulus" to each 
neuron. 
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FIG. 1: Schematic diagram of distributed signal transmission 
delays. 



A dynamical equation of the macroscopic order pa- 
rameter X{t) = ■^'^"=iXi{t) can be derived in a man- 
ner similar to what we have done for the discrete time 
model [1] in parallel with Amari's derivation for the 
synchronous update rule [l3]: A mean field exerted on 
X{t) is given by the difference of ratios of positive and 
negative inputs. Using the distribution pt{v) of inputs 
{vi = X]J=i ~ + ^i} time t, the macro- 

scopic equation can be represented as 



Pt{v)dv 



Pt{v)dv. (2) 



Under the assumption of the statistical independence 
of {xj{t — di_j)} from {wij}, the central limit theorem 
holds for the summed inputs {X]j=i ~ If 

in addition {si} are normally distributed, the distribu- 
tion pt {v) can be approximated as Gaussian characterized 
solely by the mean and the variance at time t. Then, the 
macroscopic state equation is obtained as 



are fixed to each transmission line from jth neuron to 



where F{x) is the error function 



2 dx. 



(3) 



(4) 
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and / is the ratio of the mean and the standard deviation 
of inputs {vi} to neurons 



/ = 



(5) S 



where fj.^, fJ^s, cr^ are respectively the means and 

variances of {wij} and {sj}. 

Under the assumption that microscopic states {xi} are 
statistically independent from delays {dij}, the mean 
past activity {x} = ^j- X^ILi Z]J=i ^ji^-dzj) can be rep- 
resented by the macroscopic order parameter Q as 



i=i j=i 



gis)X{t- s)ds, (6) 



where 17(5) represents the distribution of delays, (a;^) = 
:^J27=i ~ di,j) remains close to unity, if indi- 

vidual microscopic states are always approaching swiftly 
either of ±1. 

If n((T^ -I- /i^) ^ o-j , or if (x^) can be approximated as 
unity, then the dynamical equation ^ is closed with the 
macroscopic order parameter X(t). Furthermore, if the 
model parameters satisfy <C na-^ + cr^, the evolution 
equation simplifies to 



_dX{t) 
dt 



-X{t) + F[ W 



]{s)X{t - s)ds + S ,(7) 



and S — fi. 



where W — nfiw/y'na'^ 
Note that n/it^ <^ na'^ + a1 is not an essential condi- 
tion for a macroscopic equation ([7|) to hold but is merely 
introduced to make the analysis simpler. 

With this integro-differential equation, we investigate 
the influence of the dispersed delays on the macroscopic 
stability of the system. We adopt here the gamma dis- 
tributed delays. 



T{k)T \ T 



(8) 



which is characterized by the scale factor T representing 
the average delay, and the shape factor k representing 
(inversely related to) the dispersion of delays. T{k) is 
the gamma function defined by /g x'^^^e~^dx. 

Given a macroscopic stationary state X{t) = Xg that 
satisfies Xq = F {WXq + S), we examine the linear sta- 
bility with respect to the deviation from the stationary 
state, by putting X{t) — = exp(At/T). The char- 
acteristic equation for the linearized integro-differential 
equation is obtained as 

(1 + A)(l + Ar/TK)« =/3, (9) 

where (3 represents the slope of the response function, 

_ dF{WX + S) 



(3 



dX 



(10) 




FIG. 2: The gamma distributions of identical means (T = 1) 
with different shape factors; Exponentially distributed (k = 
1), with the standard deviation identical to the mean; More 
dispersed (k < 1), with the standard deviation greater than 
the mean; Less dispersed (k > 1), with the standard deviation 
smaller than the mean. 



For positive /?, the system exhibits instability, A > 0, 
if the response function F{WX -\- S) has a slope greater 
than unity, j3 > 1. In this case, the system eventually at- 
tains one of stable stationary states due to the nonlinear 
saturation. 

A dynamical instability leading to oscillation may take 
place for negative /?. In this case, the linear-stability 
boundary is obtained by solving a pair of simultaneous 
equations that represent the condition for the charac- 
teristic equation ^ to have a pure imaginary solution 
A = iio: 



arctan(aj) -I- k arctan(ra;/TK) — tt, 

/32 = {l+Uj''){l + {Tuj/TKfT. 



X=Xo 



(11) 
(12) 

Equation pT|) has a solution only if k > 1 . In the limit 
of T/t ^ 0, o; is obtained as 

oj « (kt/T) tan (7r/2K) , (13) 

and critical (3 is obtained from Eq. p^ as 

(3 « -(kt/T) tan(7r/2K)(H-tan2(7r/2K))'=/^ (14) 

In the opposite extreme of T/t — > 00, uj is obtained as 

uj « (kt/T) tan (tt/k) , (k > 2), (15) 
UJ « tan(7r(l-K/2)), (2 > k > 1), (16) 

with which the critical /3 is obtained respectively as 

f3 w -(l + tan2(7r/K))'=/^ (« > 2), (17) 
13 w -{(T/tk) tan(7r(l-K/2))}'=/2 

X {l-htan2(7r(l-K/2))}i/2, (2 > k > 1). (18) 

Figure [3] depicts the stability boundaries in the phase 
space of {T/t, — /?} obtained by numerically solving 
Eas. (fTT|l and for several shape parameters k: With 
identical delays (k —> 00), the critical coupling strength 
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FIG. 3: The stability boundaries in the space of {T/t, — /3} 
obtained for several shape parameters, k = 1.5, 2, 3, 10000. 



1/3 1 decreases monotonically with the average delay T. In 
other words, the systeni beconies more fragile with de- 
lays. This fact is consistent with the knowledge that the 
transmission delay is a destabilizing factor. It is notable 
that a system with dispersed delays exhibits reentrant 
phenomena; the stability is once lost but then recovered 
as the average delay T is increased: With the delays of 
small dispersion (2 < k < cx)), the critical |/3| rebounds 
and then saturates to a finite value. In a middle range of 
dispersion (1 < k < 2), the critical \(3\ takes a minimum 
and diverges with T. With the delays highly dispersed 
(k < 1), the network never destabilizes. 

Next, we numerically solve the macroscopic evolution 
equation ([7]) to see if there is nontrivially coexisting os- 
cillation or chaos in the parameter range in which the 
linear stability is confirmed. For this purpose, we have 
tested a number of random initial conditions of various 
power spectra; from white (jagged) to colored (smooth) 
random temporal patterns. 

Solving an integro-differential equation is generally a 
hard computational task. In some particular conditions, 
however, the computational complexity can be reduced 
drastically by devising efficient algorithms: For the ex- 
ponential distribution of delays (k = 1), the mean past 
activity Jp e^*X(t — s)ds is represented by 



(19) 



where A is a unit step. Due to the exponential kernel. 
At can be obtained by simply iterating the recurrence 
equation: 



At = e-^At-A+X{t)x 



(20) 



In addition, for the case of k = 2, the computational 
complexity could be reduced by utilizing the relation of 



-t _ -(l+e)t 



with sufficiently small 



(21) 



To the extent we have exhausted, we have not found 
any non-trivial coexisting dynamical orbits in the param- 
eter range that the linear stability is guaranteed. Figure 
m shows an amplitude of X{t) obtained for the systems 
with shape parameters of k = 2 and 1. In the case of 
K = 2, a significant amplitude of X{t) can be observed 
in the interval of mean delay T in which the system is 
linearly unstable. For the shape parameter k = 1 with 
which the system is linearly stable, the amplitude is neg- 
ligibly small in the whole range of T. 




FIG. 4: The amplitude of X{t) numerically obtained for the 
cases of W = —25 and S = Q, {{3 = —20); The linear insta- 
bility boundaries for the case of «: = 2 are T/t = 0.254 and 
15.7 and depicted by the arrows. They coincide with the crit- 
ical points at which the system numerically shows reentrant 
stability; The system with delays exponentially distributed 
(k = 1) always remains stable. 



What is the key factor in the perfect stability? As the 
system shows a perfect stability in the absence of delay, 
we suspect if a fraction of instantaneous signal transmis- 
sions lead to the stability for k < 1. We examine the 
stability of a system with the delay distribution com- 
posed of two delta functions peaked at zero and finite 
delays, a5{t) -I- (1 — a)5{t — T). It is found from the char- 
acteristic equation that the system can be destabilized 
even if delay-less lines are present in a finite fraction, 
< a < 1/2. This fact demonstrates that the presence 
of instantaneous signal transmissions alone does not nec- 
essarily induce a perfect stability. 

Next, we suspect if the long tail of the delay distribu- 
tion has led to the stability. We examine whether or not 
a system remains stable even if a lag is added to gamma 
distributed delays ([5|) as 0{t — e)g{t — e), where 9{x) is 
the Heaviside step function: 9{x) = 1 if a; > and — 
otherwise. In the presence of a lag, e > 0, the instability 
condition Eq. pTjl is modified as 

arctan(a;) + k arctan(rcLi/rK) + ecj/r = tt. (22) 

This characteristic equation possesses a solution even for 
the case of a high dispersion, n < 1. This implies that the 
perfect stability can be destroyed by a time lag. As long 
as the time lag e is small, however, the system exhibits an 
instability at a very high frequency of the order of ttt/e, 
and the amplitude of the order parameter X[t) cannot 
grow large due to the nonlinearity of the system. This 
point is verified by directly solving the original nonlinear 
macroscopic evolution equation. Figure [5] compares the 
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order parameters X{t) computed for the several kinds 
of delay distributions: the system of k = 2 may exhibit 
a large amplitude X{t); the system of k = 1 has never 
yielded a significant amplitude X{t); the system with 
gamma distributed delays of k = 1 accompanied by a 
small time lag e/r = 0.01 has yielded an order parameter 
X{t) rapidly oscillating with a small amplitude. 




t/T 

FIG. 5: The macroscopic order parameters X{t) numerically 
obtained for the cases of = -1250 and 5 = 0, (/3 = -1000), 
and T/t — 1; Solid line: Oscillation observed for the shape 
parameter k = 2; Dashed line: Stability observed for k = 1; 
Dotted line: Rapid oscillation of small amplitude observed 
for the gamma distributed delays of k = 1 accompanied by a 
time lag of e/r = 0.01. 



In the present study, we examined the stability of a 
neural network whose signal transmission delays are dis- 
tributed in time. The network is found to exhibit a reen- 
trant stability for the delays less dispersed than the ex- 
ponential distribution. The network attains a perfect 
stability for the highly dispersed delays. It is notewor- 
thy that Eurich et al also proved that the perfect stabil- 
ity is attained in the limit of highest dispersion, k = 0, 
for an ecological feedback system Q. In this Letter, we 
have revealed that the perfect stability is manifested in 
a finite range of the dispersion or the shape parameter, 
< K < 1, for a neural network. It is desirable to exam- 
ine the generality of the present findings; whether or not 
the perfect stability is achieved solely due to the delay 
distribution, irrespective of detailed dynamics of individ- 
ual elements. 
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